feat(run): emit tool_catalog NDJSON event at session start#88
Conversation
Emits a single `tool_catalog` event immediately after `session_start` and
before the first model turn so the server-side completion gate can structurally
verify that required MCP tools (e.g. `record_finding`) were actually exposed to
the model — without string-matching prose output.
Changes:
- `src/cli/cmd/tool-catalog.ts` (new): `buildToolCatalogItems()` collects
builtin tool IDs via `ToolRegistry.ids()` and MCP entries via the new
`MCP.toolEntries()`, returning `{ tools, skills }` with injectable deps for
unit testing.
- `src/mcp/index.ts`: adds `MCP.toolEntries()` — lightweight sibling to
`tools()` that returns `{ toolKey, serverName }[]` without building full
AI-SDK tool objects; used by `buildToolCatalogItems()`.
- `src/skill/skill.ts`: extends `Skill.Info` with optional `version` field
threaded from SKILL.md frontmatter; `null` when not declared.
- `src/cli/cmd/run.ts`: `await buildToolCatalogItems()` + `emit("tool_catalog",
...)` inserted between `session_start` and the first `loop()` call.
- `test/tool-catalog.test.ts` (new, TDD): 9 tests covering builtin/mcp
tool shape, the "tool exposed vs missing" consumer gate, skill version
presence/absence, and skills/tools separation.
- `EVENTS.md`: documents the new event schema including the version gap note.
Skill version gap: `Skill.Info.version` is now threaded through from
frontmatter. Skills without a `version` field in their SKILL.md emit
`version: null`.
Closes #85
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0187MsfK1upr6K2BKVbmaebQ
…capability (#88) Instead of a one-off `version` field bolt-on, `Skill.Info` now carries a `metadata: Record<string, string>` field that retains every SKILL.md frontmatter key beyond `name`/`description`, with values string-coerced (handles YAML numbers like `version: 1.0` → `"1.0"`). `version` becomes a derived convenience accessor (`metadata.version`) rather than a parallel extraction path. The dual parse path is gone — one coherent pass builds `metadata`, then derives `version` from it. Changes: - `Skill.Info`: add `metadata: z.record(z.string(), z.string()).default({})`; keep `version: z.string().optional()` as derived accessor - Loader: replace separate `"version" in md.data` extraction with a single `Object.fromEntries(…filter name/description…map String)` pass - Tests: add "non-version frontmatter keys survive in skill.metadata" test (license+author keys survive; name/description excluded; version still resolves via metadata.version) — written RED first, then GREEN `tool_catalog` event payload unchanged — still emits `version` sourced from `metadata.version`. `metadata` not added to event payload (kept lean per spec). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0187MsfK1upr6K2BKVbmaebQ
Code reviewVerdict: Address the major findings before merging. · 🔴 0 · 🟠 2 · 🟡 3 · ⚪ 0 · 0/5 resolved
🤖 Fix all 5 open findings with your agent📋 Out-of-diff findings (5)
Reviewed 6 files · 0 inline · view all 5 findings ↗ aictrl · AI code review for fast-moving teams · aictrl.dev |
- run.ts: guard buildToolCatalogItems on args.format==="json" so MCP listTools + skill scan are skipped for interactive sessions (#88 🟡) - run.ts: log catalog build failures in catch instead of silencing them so a missing tool_catalog event is diagnosable (#88 🟠) - mcp/index.ts: extract mcpToolKey() helper shared by tools() and toolEntries() — single source of truth for key derivation (#88 🟡) - EVENTS.md: document that builtin tools[] reflects the instance-level superset, not the per-model filtered dispatch set (#88 🟠) - test: regression tests for error propagation and key sanitization Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0187MsfK1upr6K2BKVbmaebQ
Review response — PR #88All 5 findings triaged. 2 fixed directly, 1 fixed via EVENTS.md correction, 1 fixed by extracting shared key helper (🟡 duplication), 1 deferred (🟡 double-listTools — substantially mitigated by the JSON-mode guard). Issues addressed (pushed to this PR)
Review claims verified false (no change needed)None — all 5 findings verified TRUE. Not addressed here
|
Code reviewVerdict: Looks good — only minor / nit comments below. · 🔴 0 · 🟠 0 · 🟡 3 · ⚪ 2 · 0/5 resolved
🤖 Fix all 5 open findings with your agent📋 Out-of-diff findings (5)
Reviewed 6 files · 0 inline · view all 5 findings ↗ aictrl · AI code review for fast-moving teams · aictrl.dev |
- EVENTS.md: reorder tool_catalog section after session_start to match actual emission order (bot finding: section placed before session_start) - run.ts: wrap buildToolCatalogItems() with 10s timeout so a hanging MCP server cannot stall session start indefinitely; on failure emit a structured tool_catalog_error event to stdout instead of console.error to stderr, so JSON consumers can distinguish failure from empty catalog - skill.ts: add comment clarifying that metadata/version are type-only fields not validated by the Info.pick() Zod parse — both are populated via manual Object.fromEntries coercion after parse succeeds - test: add 3 regression tests for timeout, structured error path, and catalog rejection propagation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0187MsfK1upr6K2BKVbmaebQ
Round 2 code review triage — PR #88 (tool_catalog session_start event)Verdict: 4 FIX · 1 DEFER · 0 IGNORE Fixes committed in Finding-by-finding
Test results
|
Summary
tool_catalogNDJSON event immediately aftersession_startand before the first model turn, listing every resolved builtin tool, MCP tool, and skill with version.MCP.toolEntries()— a lightweight sibling toMCP.tools()that returns{toolKey, serverName}[]without building full AI-SDK tool objects.versionfrom SKILL.md frontmatter throughSkill.Info(new optional field;nullin the event when not declared).src/cli/cmd/tool-catalog.tswith injectable deps for testing; 9 TDD tests cover builtin/mcp shape, the consumer gate ("tool exposed vs missing"), skill version presence/absence, and tools/skills separation.EVENTS.mdupdated with full schema documentation.Test plan
bun test test/tool-catalog.test.ts— 9/9 pass (RED→GREEN TDD)bun testfull suite — 1294 pass, 0 failbun run typecheck(turbo, all packages) — cleanSkill version gap
Skill.Info.versionis now threaded through from the SKILL.md frontmatterversionfield. Skills that don't declare a version emit"version": null. No fabricated versions — if a skill pack needs version tracking, addversion: x.y.zto its frontmatter.Closes #85
Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_0187MsfK1upr6K2BKVbmaebQ